- Israel's reported use of AI in its war against Hamas is highlighting many of the problems concerning future warfare.
- Inaccuracy and lack of meaningful human oversight could lead to errors and tragedy.
- There are military benefits to AI, but the tools to keep it in check aren't coming fast enough.
Artificial intelligence is playing a key and, by some accounts, highly disturbing role in Israel's war in Gaza.
Recent investigative reports suggest the Israeli military let an AI program take the lead on targeting thousands of Hamas operatives in the early days of the fighting and may have played a part in rash and imprecise kills, rampant destruction, and thousands of civilian casualties. The IDF flatly rejects this assertion.
The reporting offers a terrifying glimpse into where warfare could be headed, experts told Business Insider, and a clear example of how bad things can get if humans take a back seat to new technology like AI, especially in life-or-death matters.
"It's been the central argument when we've been talking about autonomous systems, AI, and lethality in war," Mick Ryan, a retired Australian major general and strategist focusing on evolutions in warfare, told BI. "The decision to kill a human is a very big one."
Earlier this month, a joint investigation by +972 Magazine and Local Call revealed Israel's Defense Force had been using an AI program named "Lavender" to generate suspected Hamas targets on the Gaza Strip, citing interviews with six anonymous Israeli intelligence officers.
The report alleges the IDF heavily relied on Lavender and essentially treated its information on who to kill "as if it were a human decision," sources said. Once a Palestinian was linked to Hamas and their home was located, sources said, the IDF effectively rubber-stamped the machine decision, barely taking more than a few seconds to review it themselves.
The speed of Israel's targeting put little effort into trying to reduce the harm to civilians nearby, the joint investigation found.
Last fall, details of Israel's Gospel program came to light, revealing that the system took Israel's target generation ability from roughly 50 a year to more than 100 each day.
When asked about the report on Lavender, the IDF referred BI to a statement posted on X by IDF spokesperson Lt. Col. (S.) Nadav Shoshani, who wrote last week that "The IDF does not use AI systems that choose targets for attack. Any other claim shows lack of sufficient knowledge of IDF processes."
Shoshani characterized the system as a cross-checking database that "is designed to aid human analysis, not to replace it." But there are potential risks all the same.
Israel isn't the only country exploring the potential of AI in warfare, and this research is coupled with increasing focus on the use of unmanned systems, as the world is frequently seeing in Ukraine and elsewhere. In this space, anxieties over killer robots are no longer science fiction.
"Just as AI is becoming more commonplace in our work and personal lives, so too in our wars," Peter Singer, a future warfare expert at the New America think tank, told BI, explaining that "we are living through a new industrial revolution, and just like the last one with mechanization, our world is being transformed, both for better and for worse."
AI is developing faster than the tools to keep it in check
Experts said that Israel's reported use of Lavender raises a host of concerns that have long been at the heart of the debate on AI in future warfare.
Many countries, including the US, Russia, and China, have been prioritizing the implementation of AI programs into their militaries. The US' Project Maven, which has since 2017 made major strides to assist troops on-the-ground by sifting through overwhelming amounts of incoming data, is just one example.
The technology, however, has often developed at faster pace than governments can keep up.
According to Ryan, the general trend "is that technology and battlefield requirements are outstripping the consideration of the legal and ethical issues around the application of AI in warfare."
In other words, things are moving too quickly.
"There's just no way that current government and bureaucratic systems of policymaking around these things could keep up," Ryan said, adding that they may "never catch up."
Last November, many governments raised concerns at a United Nations conference that new laws were needed to govern the use of lethal autonomous programs, AI-driven machines involved in making decisions to kill human beings.
But some nations, particularly ones who are currently leading the way in developing and deploying these technologies, were reluctant to impose new restrictions. Namely, the US, Russia, and Israel all appeared particularly hesitant to support new international laws on the matter.
"Many militaries have said, 'Trust us, we'll be responsible with this technology,'" Paul Scharre, an autonomous weapons expert at the Center for New American Security, told BI. But many people are not likely to trust a lack of oversight, and the use of AI by some countries, such as Israel, doesn't give much confidence that militaries are always going to use the new technology responsibly.
A program such as Lavender, as it has been reported, doesn't sound like science fiction, Scharre said, and is very in line with how global militaries are aiming to use AI.
A military would be "going through this process of collecting information, analyzing it, making sense of it, and making the decisions about which targets to attack, whether they're people as part of some insurgent network or organization, or they could be military objectives like tanks or artillery pieces," he told BI.
The next step is moving all of that information into a targeting plan, linking it to specific weapons or platforms, and then actually acting on the plan.
It's time-consuming, and in Israel's case, there's likely been a desire to develop a lot of targets very quickly, Scharre said.
Experts have expressed concerns over the accuracy of such AI targeting programs. Israel's Lavender program reportedly pulls data from a variety of information channels, such as social media and phone usage, to determine targets.
In the +972 Magazine and Local Call report, sources say the program's 90% accuracy rate was deemed acceptable. The glaring issue there is the remaining 10%. That's a substantial number of errors given the scale of Israel's air war and the significant increase in available targets provided by AI.
And the AI is always learning, for better or for worse. With every use, these programs gain knowledge and experience that they then employ in future decision-making. With an accuracy rate of 90%, as the reporting indicates, Lavender's machine learning could be reinforcing both its correct and incorrect kills, Ryan told BI. "We just don't know," he said.
Letting AI do the decision-making in war
Future warfare could see AI working in tandem with humans to process vast amounts of data and suggest potential courses of action in the heat of battle. But there are several possibilities that could taint such a partnership.
The gathered data could be too much for humans to process or understand. If an AI program is processing massive amounts of information to make a list of possible targets, it could reach a point where humans are quickly overwhelmed and unable to meaningfully contribute to decision-making.
There's also the possibility of moving too quickly and making assumptions based on the data, which increases the likelihood that mistakes are made.
International Committee Red Cross Military and Armed Group Adviser Ruben Stewart and Legal Adviser Georgia Hinds wrote about such a problem back in October 2023.
"One touted military advantage of AI is the increase in tempo of decision-making it would give a user over their adversary. Increased tempo often creates additional risks to civilians, which is why techniques that reduce the tempo, such as 'tactical patience,' are employed to reduce civilian casualties," they said.
In the quest to move quickly, humans could take their hands off the wheel, trusting the AI with little oversight.
According to the +972 Magazine and Local Call report, AI-picked targets were only reviewed for about 20 seconds, typically just to ensure the potential kill was male, before a strike was authorized.
The recent reporting raises serious questions about to what extent a human being was "in the loop" during the decision-making process. According to Singer, it's also a potential "illustration of what is sometimes known as 'automation bias,'" which is a situation "where the human deludes themselves into thinking that because the machine provided the answer, it must be true."
"So while a human is 'in the loop,' they aren't doing the job that is assumed of them," Singer added.
Last October, UN Secretary-General António Guterres and the President of the International Committee of the Red Cross, Mirjana Spoljaric, made a joint call that militaries "must act now to preserve human control over the use of force" in combat.
"Human control must be retained in life and death decisions. The autonomous targeting of humans by machines is a moral line that we must not cross," they said. "Machines with the power and discretion to take lives without human involvement should be prohibited by international law."
But while there are risks, AI could have many military benefits, such as helping humans process a wide range of data and sources in order to allow them to make informed decisions, as well as survey a variety of options for how to handle situations.
A meaningful "human in the loop" cooperation could be useful, but at the end of the day, it comes down to the human holding up their end of such a relationship — in other words, retaining authority and control of the AI.
"For the entirety of human existence, we've been tool and machine users," Ryan, the retired major general, said. "We are the masters of machines, whether you're piloting aircraft, driving a ship or tank."
But with many of these new autonomous systems and algorithms, he said, militaries won't be using machines, but rather "partnering with them."
Many militaries aren't prepared for such a shift. As Ryan and Clint Hinote wrote in a War on the Rocks commentary earlier this year, "in the coming decade, military institutions may realize a situation where uncrewed systems outnumber humans."
At present, the tactics, training, and leadership models of military institutions are designed for military organizations that are primarily human, and those humans exercise close control of the machines," they wrote.
"Changing education and training to prepare humans for partnering with machines — not just using them — is a necessary but difficult cultural evolution," they said. But that remains a work in progress for many militaries.